Learning Low-Precision Structured Subnetworks Using Joint Layerwise Channel Pruning and Uniform Quantization

نویسندگان

چکیده

Pruning and quantization are core techniques used to reduce the inference costs of deep neural networks. Among state-of-the-art pruning techniques, magnitude-based algorithms have demonstrated consistent success in reduction both weight feature map complexity. However, we find that existing measures neuron (or channel) importance estimation for such procedures at least one two limitations: (1) failure consider interdependence between successive layers; and/or (2) performing a parametric setting or by using distributional assumptions on maps. In this work, demonstrate rankings output neurons given layer strongly depend sparsity level preceding layer, therefore, naïvely estimating drive will lead suboptimal performance. Informed observation, propose purely data-driven nonparametric, channel strategy works greedy manner based activations previous sparsified layer. We our proposed method effectively combination with statistics-based generate low precision structured subnetworks can be efficiently accelerated hardware platforms as GPUs FPGAs. Using algorithms, increased performance per memory footprint over solutions across range discriminative generative

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Learning vector quantization for heterogeneous structured data

In this paper we introduce an approach to integrate heterogeneous structured data into a learning vector quantization. The total distance between two heterogeneous structured samples is defined as a weighted sum of the distances in the single structural components. The weights are adapted in every iteration of learning using gradient descend on the cost function inspired by Generalized Learning...

متن کامل

Learning Sparse Structured Ensembles with SG-MCMC and Network Pruning

An ensemble of neural networks is known to be more robust and accurate than an individual network, however usually with linearly-increased cost in both training and testing. In this work, we propose a two-stage method to learn Sparse Structured Ensembles (SSEs) for neural networks. In the first stage, we run SG-MCMC with group sparse priors to draw an ensemble of samples from the posterior dist...

متن کامل

the relationship between using language learning strategies, learners’ optimism, educational status, duration of learning and demotivation

with the growth of more humanistic approaches towards teaching foreign languages, more emphasis has been put on learners’ feelings, emotions and individual differences. one of the issues in teaching and learning english as a foreign language is demotivation. the purpose of this study was to investigate the relationship between the components of language learning strategies, optimism, duration o...

15 صفحه اول

Ensemble Pruning Using Reinforcement Learning

Multiple Classifier systems have been developed in order to improve classification accuracy using methodologies for effective classifier combination. Classical approaches use heuristics, statistical tests, or a meta-learning level in order to find out the optimal combination function. We study this problem from a Reinforcement Learning perspective. In our modeling, an agent tries to learn the b...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Applied sciences

سال: 2022

ISSN: ['2076-3417']

DOI: https://doi.org/10.3390/app12157829